Assessing the performance of classifiers when classes arise from a continuum

The situation where classes arise by dividing the range of a continuous response variable into intervals is discussed. The focus is on assessing the performance of classi'ers. Due to the underlying continuum, all misclassi'cations are not equally grave. The probability of misclassi'cation (pmc) is not optimal in this situation. An alternative performance measure, the squared error rate (sqerr) is proposed. It is related to the mean squared error of regression, and penalises misclassi'cations according to their severity. Also, because of measurement errors in the response variable, there are misallocated class labels in data sets used for training and testing. Estimates of the pmc and the sqerr are developed for this situation. The estimates are tested and compared on a real data set and in a simulation. c © 2003 Elsevier B.V. All rights reserved.